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[0001] This application is based on Japanese Patent Application No. 
2002-204557 filed with Japan Patent Office on July 12, 2002, the entire 
content of which is hereby incorporated by reference. 
BACKGROUND OF THE INVENTION 
Field of the Invention 

[0002] The present invention relates to image processing methods, and 
more particularly, to an image processing method suitable for image 
correction. 

Description of the Related Art 

[0003] In order to convert text information, diagram information, 
picture information and the like on a recording medium (document) such as 
a paper sheet into electronic data, the document is generally read through 
an image input apparatus such as an image scanner or a digital camera. 
[0004] In the image processing apparatus disclosed in the 
aforementioned Japanese Laid-Open Patent Publication No. 2000-36902, 
the operation was tedious since the operator had to determine the 
extraction area. Also, there was a possibility that an unnecessarily large 
area of the extraction area may be specified. In such a case, image 
information including a large amount of useless area will be extracted. 
This means that a large amount of information not required is included, 
leading to image information of a large amount being extracted. This 
image processing apparatus requires the operator to designate the 
extraction area. The position of the information in the extraction area 
depends on the manipulation of the operator. There is a possibility that 
information in the extracted image will be disproportioned in one direction. 
SUMMARY OF THE INVENTION 

[0005] An object of the present invention is to provide an image 
processing method that can prevent loosing information from image 
information subjected to rotational correction, and that can minimize the 
amount of data in the image information subjected to rotational correction. 
[0006] An image processing method comprising a first step of detecting 
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a data region in input image data, a second step of rotating the input image 
data in accordance with inclination of an image within the input image 
data, a third step of detecting whether the rotated data region protrudes 
from the input image data, and a fourth step of extracting the smallest area 
including the rotated data region when protrusion is detected at the third 
step. 

[0007] The foregoing and other objects, features, aspects and 
advantages of the present invention will become more apparent from the 
following detailed description of the present invention when taken in 
conjunction with the accompanying drawings. 
BRIEF DESCRIPTION OF THE DRAWINGS 

[0008] Fig. 1 shows an example of a structure of an image processing 
system according to an embodiment of the present invention. 
[0009] Fig. 2 is a functional block diagram of an image processing 
apparatus according to a first embodiment. 

[00 10] Fig. 3 is a flow chart of image processing carried out by the 
image processing apparatus of the first embodiment. 
[0011] Figs. 4A and 4B are density histograms to describe the under 
color removal process carried out by the image processing apparatus of the 
first embodiment. 

[0012] Fig. 5 is a flow chart of a region extraction process carried out by 
the image processing apparatus of the first embodiment. 
[00 13] Fig. 6 shows an example of dividing image information into a 
plurality of regions in the region extraction process carried out by the 
image processing apparatus of the first embodiment. 

[00 14] Fig. 7 shows an example of a setting screen to set a condition to 
extract a specific region by the image processing apparatus of the first 
embodiment. 

[0015] Fig. 8 is a flow chart of an image inclination detection process 
carried out by the image processing apparatus of the first embodiment. 
[00 16] Fig. 9 shows an example of a process of detecting an inclination 
detection region including many edge components from a binary image. 
[0017] Figs. 10A and 10B are cumulative histograms representing the 



number of pixels whose value of the detection angle direction for each line 
is 1. 

[0018] Fig. 11 is a flow chart of an extraction area determination 
process carried out by the image processing apparatus of the first 
embodiment. 

[0019] Figs. 12A-12D are diagrams to describe image processing carried 
out by the image processing apparatus of the first embodiment. 
[0020] Figs. 13A-13D are diagrams to describe a specific region 
extraction process carried out by a modification of the image processing 
apparatus of the first embodiment. 

[002 1] Fig. 14 represents a relative position of a candidate region with 
respect to image information. 

[0022] Figs. 15A-15D are diagrams to describe image processing carried 
out by a modification of the image processing apparatus of the first 
embodiment. 

[0023] Fig. 16 is a functional block diagram of an image processing 
apparatus according to a second embodiment. 

[0024] Fig. 17 shows a relative position of a specific region with respect 
to a candidate area. 

[0025] Fig. 18 is a flow chart of an extraction area determination 
process carried out by the image processing apparatus of the second 
embodiment. 

[0026] Figs. 19A-19D are first diagrams to describe image processing 

carried out by the image processing apparatus of the second embodiment. 

[0027] Figs. 20A-20D are second diagrams to describe image processing 

carried out by the image processing apparatus of the second embodiment. 

[0028] Figs. 21A-21C are diagrams to describe image processing carried 

out by an image processing apparatus. 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0029] Embodiments of the present invention will be described 

hereinafter with reference to the drawings. In the drawings, the same 

reference characters represent the same or corresponding components. 

Therefore, description thereof will not be repeated. 



[0030] As an approach to prevent loosing information of a protruding 
area from an output image among the input image, Japanese Laid-Open 
Patent Publication No. 2000-36902 mentioned in Description of the Related 
Art discloses an image processing apparatus displaying an input image 
that is rotated and corrected with an extraction area in an overlapping 
manner to determine the position of the extraction area through 
designation of an operator and extracting the determined extraction area. 
[003 1] Description will be provided on the aforementioned " protruding 
area" as a preamble to the present invention based on the inventor's view. 
[0032] In the case where the document is set inclined with respect to 
the image scanner as shown in Fig. 21 A, or when the document is image- 
sensed by a digital camera in an inclined manner, text information, 
diagram information, picture information and the like included in the 
image information as electronic data obtained through the image scanner 
or digital camera may be inclined as shown in Fig. 2 IB. This inclination is 
defined by an angle referenced to a certain side of a rectangle of the image 
information. 

[0033] To correct this inclination, inclination of the information 
included in the obtained image information with respect to the image 
information is detected. Then, the image information is rotated by an 
angle of rotation corresponding to the detected inclination for conversion of 
the image information as shown in Fig. 2 1C. Thus, rotational correction is 
conducted to compensate for the inclination. 

[0034] This rotational correction was based on a predetermined 
coordinate position, for example, the centroid, of image information 
previous to rotational correction. Therefore, the image information 
subjected to rotational correction may partially be located out from the area 
of the original image information previous to rotational correction as shown 
in Fig. 21C. In the case where image information is present in the 
protruding area, information in that protruding area will be lost. There is 
also a problem that, since the position of information in the extraction area 
is not taken into account, information may be disproportioned in one 
direction in the extracted image. 



-4- 



First Embodiment 
[0035] Fig. 1 shows an example of a structure of an image processing 
system according to an embodiment of the present invention. Referring to 
Fig. 1, an image processing system of the first embodiment includes an 
image processing apparatus 100, an image input device 200, an external 
storage device 300, and an image output device 400. 

[0036] Image processing apparatus 100 is formed of a computer such as 
a personal computer (referred to as PC hereinafter). Image processing 
apparatus 100 includes a control unit to provide entire control of image 
processing apparatus 100, an input unit to enter predetermined 
information into image processing apparatus 100, an output unit to provide 
predetermined information from image processing apparatus 100, a storage 
unit to store predetermined information, and a communication unit 
identified as an interface to connect image processing apparatus 100 on a 
network. 

[0037] Image input device 200 is a scanner, a digital camera, or the like 
to pick up an image, temporarily store the image information obtained, and 
output the temporarily stored image information. Image input device 200 
is connected to image processing apparatus 100 to provide image 
information to image processing apparatus 100. 

[0038] External storage device 300 is a FDD (Floppy (R) Disk Drive), 
HDD (Hard Disk Drive), CD (Compact Disk) drive, MO (Magneto Optical 
disk) drive or the like to read in programs and data stored in a storage 
medium 301 for transmission to image processing apparatus 100. In 
response to designation from image processing apparatus 100, required 
information such as image information processed at image processing 
apparatus 100 is written into a record medium 301. 
[0039] A computer readable recording medium 181 is a recording 
medium storing a program such as a magnetic tape, a cassette tape, a 
magnetic disk such as a floppy (R) disk or a hard disk, an optical disk such 
as a CD-ROM (Compact Disk Read Only Memory), DVD (Digital Versatile 
Disk), a magneto optical disk such as a MO, MD (Mini Disc), a memory 
card such as an IC card or an optical card, or a semiconductor memory such 
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as a mask ROM, EPROM (Erasable Programmable Read Only Memory), 
EEPROM (Electrically Erasable and Programmable Read Only Memory), or 
a flash ROM. 

[0040] Image output device 400 is a printer or the like to output image 
information onto a medium such as a paper sheet. Image output device 
400 is connected to image processing apparatus 100 to output image 
information processed by image processing apparatus 100. 
[0041] Fig. 2 is a functional block diagram of image processing 
apparatus 100 of the first embodiment. Referring to Fig. 2, image 
processing apparatus 100 includes an image size reduction unit 111 
reducing image information input from image input device 200, an under 
color removal unit 112 to remove under color from image information 
reduced at image size reduction unit 111, a region separation unit 113 to 
separate image information subjected to under color removal at under color 
removal unit 112 to a candidate region, a specific region extraction unit 114 
to extract a specific region from the candidate region separated at region 
separation unit 113, an image inclination detection unit 115 to detect 
inclination of image information subjected to a specific region extraction at 
specific region extraction region 1 14, an extraction area determination unit 
116 to determine an extraction area from image information subjected to 
inclination detection at image inclination detection unit 115 and specific 
region extraction at specific region extraction unit 114, an image rotation 
unit 117 to rotate image information input from image input device 200 by 
an angle of rotation in accordance with the inclination detected by image 
inclination detection unit 115, and an image extracting unit 118 to cut the 
extraction area determined at extraction area determination unit 1 16 from 
rotated image information rotation by image rotation unit 117. 
[0042] Image size reduction unit 111 reduces image information 
applied from image input device 200 and transmits the reduced image 
information to under color removal unit 112. Image information is 
reduced for the purpose of increasing the speed of subsequent image 
processing. Image size reduction unit 111 is dispensable in the case where 
increasing the speed of image processing is not required. 
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[0043] Under color removal unit 112 removes the under color present in 
the background region from the image information reduced at image size 
reduction unit 111 to provide image information having under color 
removed to region separation unit 113. This under color refers to lightly 
colored yellow generated when recycle paper or the like is scanned, or the 
pale under pattern color present in the background of an image. Under 
color removal of image information is conducted for the purpose of 
facilitating extraction of information such as text information, diagram 
information, picture information and the like. Under color removal is not 
required when the background of the image is uniform white. In this case, 
under color removal unit 112 is dispensable. 

[0044] Region separation unit 113 separates a candidate region from 
image information subjected to under color removal at under color removal 
unit 113. The candidate region includes a text region, a diagram region, a 
picture region, a rule mark region, a margin region, and the like. The text 
region mainly includes text information. A diagram region, picture region, 
and rule mark region mainly include diagram information, picture 
information, and rule mark information, respectively. A margin region 
refers to a region other than the diagram region, picture region, and rule 
mark region in the entire region of image information. Region separation 
unit 113 may be implemented to separate a candidate region from image 
information applied from image input device 200, or from image 
information reduced by image size reduction unit 111. 
[0045] Specific region extraction unit 114 extracts a candidate region 
that satisfies a predetermined condition from the candidate regions 
separated by region separation unit 113 as a specific region. The 
predetermined condition is defined by the attribute of the candidate region. 
The attribute of a candidate region includes a text attribute, a graphic 
attribute, a picture attribute, and a rule mark attribute when the candidate 
region is a text region, a diagram region, a picture region, and a rule mark 
region, respectively. The predetermined condition may be one of these 
attributes, or a plurality of combinations of such attributes. 
[0046] Image inclination detection unit 115 detects the inclination of 
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image information from the image information subjected to a specific region 
extraction process by specific region extraction unit 114. Image inclination 
detection unit 115 may be implemented to detect inclination of image 
information applied from image input device 200, image information 
reduced at image size reduction unit 111, image information subjected to 
under color removal at under color removal unit 112, or image information 
subjected to a candidate region separation process by region separation 
unit 113. 

[0047] Extraction area determination unit 116 determines an 
extraction area based on data of a specific region extracted by specific 
region extraction unit 114 and the inclination of image information 
detected at image inclination detection unit 115. 

[0048] Image rotation unit 117 rotates image information applied from 
image input device 200 by an angle of rotation in accordance with the 
inclination of image information detected at image inclination detection 
unit 115. Image rotation unit 117 may also rotate image information 
reduced at image size reduction unit 111, image information subjected to 
under color removal at under color removal unit 112, image information 
subjected to a specific region separation process by region separation unit 
113, or image information subjected to a specific region extraction process 
by specific region extraction unit 114. 

[0049] Image extracting unit 118 extracts the extraction area 
determined by extraction area determination unit 116 from image 
information rotated by image rotation unit 117. Image extracting unit 118 
provides the image information in the extraction area to external storage 
device 300 or image output device 400. 

[0050] Fig. 3 shows a flow chart of image processing carried out by 
image processing apparatus 100 of the first embodiment. First, image 
information represented by respective color components of RGB (Red, 
Green, Blue) is entered from image input device 200 (step Sll). The 
image information input at step Sll is subjected to a process of reducing 
the image size by image size reduction unit 111 (step S12). The image size 
can be reduced by, for example, employing the method of dividing image 
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information into a plurality of rectangular regions, and replacing respective 
rectangular regions with the average density value. In this case, the size 
of a rectangular region is to be determined based on the resolution or the 
size of the image information. Specifically, a reduction process is 
preferably applied so that the length of the shorter side of the rectangle 
corresponds to approximately 500-1000 pixels. This reduction process is 
aimed to increase the speed of subsequent image processing. Therefore, 
step S12 may be executed selectively. 

[0051] By under color removal unit 112, the image information reduced 
at step S12 is subjected to under color removal (step S13). One method of 
under color removal is, for example, enhancing the contrast of the image 
information and removing the highlight region. Step S13 may be executed 
selectively. The process of under color removal will be described 
afterwards with reference to Figs. 4A and 4B. 

[0052] Then, a region extraction process of extracting a specific region 
from image information subjected to under color removal at step S13 is 
executed (step S14). This region extraction process will be described 
afterwards with reference to Fig. 5. 

[0053] An image inclination detection process of detecting inclination of 
image information from the image information subjected to a specific region 
extraction process at step S14 is executed (step S15). This image 
inclination detection process will be described afterwards with reference to 
Fig. 8. 

[0054] Based on the specific region extracted at step S14 and the 
inclination of image information detected at step S 15, the extraction area to 
be extracted from image information subjected to rotation, when image 
information is rotated, is determined (step S16). This extraction area 
determination process will be described afterwards with reference to Fig. 11. 
[0055] The image information input at step Sll is rotated by an angle 
of rotation in accordance with the inclination of image information detected 
at step S15 (step S17). The extraction area determined at step S16 is 
extracted from rotated image information (step S18). Image information 
in the extraction area is output to external storage device 300 or image 



output device 400 (step S19). 

[0056] The image rotation process executed at step S17 can be executed 
prior to the extraction area determination process of step S16. 
[0057] Figs. 4A and 4B are density histograms to describe the under 
color removal process carried out by image processing apparatus 100 of the 
first embodiment. The under color removal process is a process executed 
at step S13 in the image processing procedure of Fig. 3. Fig. 4A is a 
density histogram prior to an expansion process. Fig. 4B is a density 
histogram after an expansion process is applied. 

[0058] First, a density histogram of respective RGB components of the 
image information is produced (Fig. 4A). A density conversion process of 
expanding in the largest value direction and smallest value direction in 
* each histogram is carried out (Fig. 4B). In the case where the expanded 
result exceeds the largest value and smallest value of a predetermined 
density gradation, expanded data corresponding to that pixel will take the 
largest value or smallest value of the density gradation. By setting a 
larger amount of expansion in the largest value direction, a highlight 
region can be removed. Accordingly, under color removal is effected. 
Furthermore, instead of subjecting the entire image information to the 
same under color removal process, the image information can be divided 
into a plurality of rectangular regions, and carry out under color removal 
with a different amount of expansion for each rectangular region. 
Accordingly, under color removal process can be applied irrespective of the 
gray level of the background color or background pattern. 
[0059] Fig. 5 is a flow chart of a region extraction process carried out by 
image processing apparatus 100 of the first embodiment. The region 
extraction process is executed at step S14 in the image processing 
procedure of Fig. 3. Referring to Fig. 5, image information subjected to 
under color removal at step S13 is divided into a plurality of rectangular 
regions by region separation unit 113 (step S21). 

[0060] Fig. 6 shows an example of image information divided into a 
plurality of regions in the image extraction process carried out by image 
processing apparatus 100 of the first embodiment. Image information 
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600A is divided into a plurality of rectangular regions. A rectangular 
region 600C is an enlargement of one of divided rectangular regions 600B. 
In the present embodiment, each block of a rectangular region has the size 
of 8 x 8 pixels. The size of one block preferably corresponds to the size of 
substantially one character to facilitate discrimination between a text 
region and a rule mark region. 

[006 1] Returning to the flow chart of Fig. 5, the steps set forth below 
(step S22) are executed for each of all the rectangular regions divided at 
step S21. First, a color value histogram of the rectangular region is 
calculated (step S23). Then, determination is made whether the 
histogram distribution is uniform for all the color values (step S24). When 
uniform, determination is made that the rectangular region is a picture 
region, and control proceeds to step S36 (step S25). When not uniform, 
control proceeds to step S26. 

[0062] Determination is made whether the histogram distribution is 
concentrated at the highlight region (step S26). When concentrated at the 
highlight region, determination is made that the rectangular region is a 
margin region, and control proceeds to step S36 (step S27). Determination 
of whether the histogram distribution is concentrated at a highlight region 
or not is based on, for example, whether the highlight region is 
concentrated at a region with a color value of at least 230 in the case where 
the color value is set in the range of 0-255. When the distribution is not 
concentrated at a highlight region, control proceeds to step S28. At step 
S28, the number of color pixels included in the rectangular region is 
calculated. A color pixel is a pixel whose chroma component exceeds a 
predetermined value. When the number of color pixels includes in the 
rectangular region is equal to or exceeds a predetermined value, 
determination is made that the rectangular region is a diagram region (step 
S31), and control proceeds to step S36. When the number of color pixels 
included in the rectangular region is below the predetermined value, 
control proceeds to step S32. 

[0063] Then, a linking component of pixels other than white pixels 
included in the rectangular region is extracted (step S32). Determination 
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is made whether the linking component transverses the rectangular region 
(step S33). A linking component is a group of pixels other than white 
pixels, adjacent to each other. As an alternative to the process of 
extracting a linking component of pixels excluding white pixels, a process of 
extracting a Unking component of pixels whose color value component is 
below a predetermined value may be carried out. Also, a linking 
component formed of only black pixels, (a pixel whose color value is 0) may 
be extracted. Determination is made that the rectangular region is a rule 
mark region when the Unking component transverses the rectangular 
region. In this context, control proceeds to step S36 (step S34). In the 
case where a Unking component does not transverse the rectangular region, 
determination is made that the rectangular region is a text region, and 
control proceeds to step S36 (step S35). 

[0064] The processes of steps S22-S35 are repeated for aU the 
rectangular regions (step S36). Each rectangular region is separated into 
regions of respective attributes. Then, adjacent rectangular regions 
having the same attribute are combined. The combined rectangular 
regions are extracted as a candidate region of that attribute (step S37). 
When the majority of eight rectangular regions Ai-As adjacent in eight 
directions located vertically, horizontaUy and obUquely with respect to a 
certain rectangular region Ao has the same attribute b, the attribute of 
rectangular region Ao surrounded by the eight rectangular regions Ai-As 
having an attribute bo, not of attribute b, may be regarded as having 
attribute b. 

[0065] FinaUy, determination is made whether the candidate region 
extracted at step S37 satisfies a predetermined condition or not by specific 
region extraction unit 114 (step S38). When a predetermined condition is 
satisfied, the candidate region is extracted as a specific region (step S39). 
Step S38 and step S39 constitute a specific region extraction process. 
Then, control returns to step S14 of the image processing procedure of Fig. 
3. A predetermined condition may be the relative position of the candidate 
region with respect to the image area of the image information, or a 
condition defined by the attribute of the candidate region. The 
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predetermined condition may be set in advance, or specified by the operator 
at every input of image information. The way operator specifies that a 
candidate region is important or not will be described afterwards with 
reference to Fig. 7. The event of a predetermined condition being defined 
by the relative position of the candidate region with respect to the image 
area of image information will be described afterwards with reference to a 
modification of the first embodiment. For example, if the input image 
information corresponds to image information including the text 
information, diagram information, picture information, and rule mark 
information, the predetermined condition is to be set on the candidate 
region being a region including information other than a margin region, i.e., 
the candidate region being a text region, diagram region, photograph region, 
and rule mark region, or set on the candidate region being a region 
excluding a region including an important information from the image 
information, i.e., the candidate region being a text region, diagram region, 
and a picture region. In the case where importance is placed only on 
numerics and characters such as data of experiments, the predetermined 
condition is set on the candidate region being a text region. 
[0066] In the case where image information input from image input 
device 100 is image information of gray scale instead of image information 
including RGB components, the processes of steps S28 and S29 are 
dispensable. In this case, a diagram region and a rule mark region are 
regarded as a region of the same attribute. 

[0067] Fig. 7 shows an example of a setting screen to set the condition 
of extracting a specific region by image processing apparatus 100 of the 
first embodiment. The setting screen includes a plurality of buttons 
through GUI (Graphical User Interface) to set the condition of extracting a 
specific region. By specifying any of the plurality of buttons, the attribute 
corresponding to the depressed button is set as the condition of extracting a 
specific region from the candidate regions. The setting screen includes a 
"text/picture" button to set a text region and a picture region as the 
condition of extracting a specific region, a "text/diagram" button to set the 
condition of extracting a text region and a diagram region as a specific 
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region, a "text/rule mark" button to set the condition of extracting a text 
region and a rule mark region as a specific region, a "text" button to set the 
condition of extracting a text region as a specific region, a "picture" button 
to set the condition of extracting a picture region as a specific region, a 
"diagram" button to set the condition of extracting a diagram region as a 
specific region, and a "rule mark" button to set the condition of extracting a 
rule mark region as a specific region. Buttons to set another attribute or a 
button to set a combination of a plurality of attributes may be included in 
addition to the aforementioned buttons. 

[0068] Fig. 8 is a flow chart of the image inclination detection process 
carried out by image processing apparatus 100 of the first embodiment. 
The image inclination detection process is executed at step S15 in the 
image processing procedure of Fig. 3. 

[0069] Referring to Fig. 8, the color value component of each pixel in 
the image information is calculated to produce a color value image by image 
inclination detection unit 115 (S41). A binarization process is applied on 
the produced color value image based on a predetermined threshold value 
to produce a binary image (step S42). An example of a method of 
obtaining a threshold value includes the steps of obtaining the edge 
component from image information using a Sobel filter that is a general 
edge detection filter, obtaining the average of the color values 
corresponding to all the edge pixels, and setting the obtained average value 
as the threshold value. This method takes the advantage that there is a 
high possibility of a color value to be used as the threshold value in 
binarization being present at the boundary region of a character or a 
graphic. 

[0070] Pixels other than the specific region have the pixel value 
converted to 0 in the binarization image generated at step S42 (step S43). 
A square region including the greatest edge components in the binary 
image converted at step S43 is detected as an inclination detection region 
(step S44). 

[0071] Fig. 9 shows an example of the process of detecting an 
inclination detection region with the greatest edge components from a 
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binary image. Referring to Fig. 9, a square region including the greatest 
edge components in the binary image generated at step S42 in the image 
inclination detection process described with reference to Fig. 8 is detected 
as an inclination detection region. A region represented by a character 
generally includes relatively many edge components. Therefore, when a 
region represented by a character is included in image information, the 
region represented with a character is detected as an inclination detection 
region. Furthermore, by using a square region for the inclination 
detection region, an image inclination detection process can be effected for 
both a vertical- writing manuscript as well as a horizontal- writing 
manuscript based on the same inclination detection criterion by just 
rotating the square region 90°. The inclination detection region preferably 
has a size of approximately 500 x 500 pixels. 

[0072] Referring to Fig. 8 again, the loop of steps S45-S48 is executed 
for all detection angles at inclination angle detection unit 115 (step S45). 
A detection angle is the angle with respect to an inclination detection 
region when a cumulative histogram that will be described afterwards is to 
be produced. For example, there are 91 detection angles with the step of 
1° in the range of -45° to 45°. The loop of steps S45-S48 is executed for all 
the 91 detection angles. The number of pixels having the value of 1 is 
counted for each line in the detection angle direction in the inclination 
detection region. A cumulative histogram for the detection angle is 
generated (step S46). A relief degree UDl(s) of the cumulative histogram 
is calculated according to equation (1) (step S47). 

UD(s) = £|f(i)-f(i + l)| -(1) 

i 

[0073] Here, s represents the detection angle. Then, the loop of steps 
S45-S48 is repeated (step S48). 

[0074] Figs. 10A and 10B are cumulative histograms indicating the 
number of pixels whose value is 1 with respect to each line in the detection 
angle direction. Fig. 10A schematically shows an example of an 
inclination detection region. The arrows in Fig. 10A represents respective 
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lines of the detection angle direction. This line refers to a line of one pixel 
width transversing the inclination detection region. 
[0075] Fig. 10B shows an example of a cumulative histogram with 
respect to an inclination detection region. Referring to Fig. 10B, the 
cumulative histogram represents the frequency f (i) of pixels having a value 
of 1 present in each line (the i-th line) for all the lines in the inclination 
detection region. The relief value UDl(s) of the cumulative histogram is 
calculated by the above equation (1) using f (i). 

[0076] Returning to Fig. 8, the inclination detection region is rotated 
90° (step S49). A process similar to steps S45-S48 is executed at steps 
S51-S54. A relief value UD2(s) for each detection angle s is calculated 
according to equation (1). 

[0077] Finally, the detection angle s corresponding to the highest relief 
degree among relief values UDl(s) and UD2(s) calculated by the process of 
steps S45-S48 and steps S51-S54 is detected as the inclination of image 
information (step S55). Then, control returns to step S15 of the image 
processing procedure of Fig. 3. 

[0078] Fig. 11 is a flow chart of an extraction area determination 
process carried out by image processing apparatus 100 of the first 
embodiment. The extraction area determination process is executed at 
step S16 in the image processing procedure of Fig. 3. Referring to Fig. 11, 
the specific region extracted at step S14 of Fig. 3 is rotated about the 
centroid of image information by extraction area determination unit 116, 
based on the inclination of image information detected at step S15 of Fig. 3. 
Detection is made whether the rotated specific regions subjected to rotation 
protrudes from the image area of image information previous to rotation 
(step S61). When detection is made that the specific region protrudes, 
control proceeds to step S62, otherwise, to step S63. 

[0079] Specifically, the coordinates of the contour of the rotated specific 
region are calculated, and determination is made whether the rotated 
coordinates of the contour protrudes from the image area of image 
information previous to rotation. This process is directed to minimizing 
the number of pixels to be processed to allow high speed processing. 
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Alternatively, the smallest rectangular area surrounding the specific region 
may be obtained in advance to calculate the rotated coordinates of the four 
corners of that smallest rectangular area, and determination is made 
whether the rotated coordinates of the four corners of the smallest 
rectangular area protrudes from the image area of image information 
previous to rotation. Also, the rotated coordinates can be calculated for all 
pixels in the specific region to determine whether the rotated coordinates of 
all pixels in the specific region protrudes from the image area of the image 
information previous to rotation or not. 

[0080] When detection is made that the specific region protrudes, the 
smallest rectangular region including the specific region rotated at step S61 
is defined as the extraction area (step S62). Then, control proceeds to step 
S16 of the image process described with reference to Fig. 3. 
[0081] When detection is made that the specific region does not 
protrude, the image area of the image information previous to rotation is 
determined as the extraction area (step S63). Then, control returns to step 
S16 of the image processing procedure described with reference to Fig. 3. 
[0082] Figs. 12A-12D are diagrams to describe image processing carried 
out by image processing apparatus 100 of the first embodiment. Fig. 12A 
shows a document 501. Document 501 includes text information and 
diagram information. Document 501 is input as image information by 
image input device 200 and applied to image processing apparatus 100. 
The case where document 501 is input with document 501 in an inclined 
state will be described hereinafter. 

[0083] Fig. 12B corresponds to image information 502 previous to 
rotation. Image information 502 previous to rotation is input into image 
processing apparatus 100 as image information of the smallest rectangular 
area including text information and diagram information. Then, specific 
regions 511 and 512 are extracted from image information 502 previous to 
rotation. Inclination of image information is detected. Since image 
information includes text information and diagram information, the text 
region and the diagram region are extracted as specific regions when the 
condition of extracting text region 512 and diagram region 512 as specific 



- 17 - 



i 



regions is set. 

[0084] Fig. 12C shows rotated image information 503 subjected to 
rotation. Referring to Fig. 12C, when image information 502 previous to 
rotation is rotated by an angle of rotation corresponding to the inclination 
detected at Fig. 12B, determination is made whether the specific region of 
rotated image information 503 protrudes from the image area of image 
information 502 previous to rotation. Here, protrusion of a specific region 
of rotated image information 503 from the image area of image information 

502 previous to rotation is detected. 

[0085] Fig. 12D shows extracted image information 504. Since 
detection is made of protrusion in Fig. 12C, the extraction area of the 
smallest area including specific regions 511 and 512 of rotated image 
information 503 is determined. This area is extracted as extracted image 
information 504. The image area of extraction image information 504 is 
rectangular. 

[0086] In the case where a specific region of rotated image information 

503 protrudes from the image area of image information 502 previous to 
rotation, a smallest rectangular area including a specific region of rotated 
image information 503 is extracted from rotated image information. 
Therefore, loosing information from the image information subjected to 
rotation correction can be prevented. Also, the amount of data of image 
information subjected to rotation correction can be minimized. 

[0087] When the specific region of rotated image information does not 
protrude from the image area of image information previous to rotation in 
image processing apparatus 100 of the first embodiment, the image area of 
image information previous to rotation is extracted from rotated image 
information. Therefore, an image having a size identical to that of the 
input image information can be output. 

[0088] In image processing apparatus 100 of the first embodiment, a 
region of a predetermined attribute or an attribute specified by an operator 
is extracted. Therefore, loosing information of a certain attribute from 
image information subjected to rotation correction can be prevented. 
[0089] The above description of the first embodiment is based on 
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processing carried out by image processing apparatus 100. The present 
invention can be taken as image processing method executing the process of 
Fig. 3, an image processing program to cause a computer to execute the 
process of Fig. 3, and a computer readable recording medium recorded with 
an image processing program. 

[Modification of a First Embodiment] 
[0090] In the first embodiment, a specific region is extracted in 
accordance with the attribute of the region when a specific region is to be 
extracted by specific region extraction unit 114 of image processing 
apparatus 100. As a modification of the first embodiment, the condition of 
extracting a specific region by specific region extraction unit 114 is set as a 
condition defined by the relative position of a candidate region with respect 
to the image area of image information. 

[009 1] This modification of the first embodiment corresponds to an 
improvement of the specific region extraction process carried out at steps 
S38 and S39 of the region extraction process described with reference to Fig. 
5. 

[0092] Figs. 13A-13D are diagrams to describe a specific region 
extraction process carried out by a modification of the image processing 
apparatus 100 of the first embodiment. Fig. 13A represents input image 
information. The input image information includes text information, 
diagram information, and rule mark information. Rule mark information 
is located in the proximity of the upper and lower ends of the image 
information. 

[0093] Fig. 13B represents an extracted candidate region. A document 
is input as image information from image input device 200 to image 
processing apparatus 100. The region including text information, diagram 
information, and rule mark information are extracted as a text region 511, 
a diagram region 512, and a rule mark region 513, respectively. 
[0094] Fig. 13C represents the centroids of the candidate regions. 
Centroids Gi and G2 of text region 511, centroid G3 of diagram region 512, 
and centroids Go, G4 of rule mark region 513 are obtained. 
[0095] Fig. 13D represents the relative position between the centroid of 
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image information and the centroid of candidate regions. The evaluation 
value of each candidate region is calculated based on the relative position of 
a candidate region with respect to the image area of image information. 
Specifically, the evaluation value of each candidate region is calculated in 
accordance with the relative position of the centroid of a candidate region 
with respect to the centroid of the image area of the image information. 
[0096] Fig. 14 represents the relative position of a candidate region 
with respect to image information. When the centroid O of the image area 
of the image information is taken as the origin, the coordinates of centroid 
Gn of the candidate region can be represented as (a n , b n ). The length of the 
image area of image information in the horizontal direction and the vertical 
direction is set as "w" and "h", respectively. An evaluation value P n of the 
current candidate region can be calculated by, for example, equation (2). 

p looCw-aK |)(h-2ib n I) _ (2) 

wh 

[0097] By equation (2), evaluation value P n becomes higher as the 
relative position of centroid G n of the candidate region with respect to 
centroid O of the image area of the image information becomes closer. 
When centroid Gn of the candidate region overlaps with centroid O of the 
image area of the image information, evaluation value Pn = 100 is 
established. In the case where centroid G n of the candidate region is 
located at the farthest edge of the image area of image information, 
evaluation value P n = 0 is established. In other words, evaluation value P n 
is proportional to absolute value I a n I of the x coordinate and the absolute 
value I bn I of the y coordinate value of centroid Gn of the candidate region. 
[0098] Calculation of evaluation value Pn of a candidate region is not 
restricted to equation (2) set forth above. Any method that allows 
calculation of evaluation value P n by the relative position of a candidate 
region with respect to the image area of the image information can be used. 
For example, the following equation (3) can be used. 
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P n =(w-2|a n |)(h-2|b n |) -(3) 

[0099] By equation (3), evaluation value Pn = wh is established when 
centroid Gn of the candidate region overlaps with centroid O of the image 
area of the image information. When centroid G n of the candidate region 
is located at the farthest end of the image area of the image information, 
evaluation value P n = 0 is established. 

[0100] Evaluation value P n calculated by equation (2) or (3) attains a 
higher value as a function of centroid Gn of the candidate region 
approaching centroid O of the image area of the image information. 
Therefore, a candidate region located close to the centroid of the image area 
of the image information will be extracted as a specific region by setting the 
condition of an evaluation value Pn attaining at least a certain value. 
[0101] Evaluation value P n calculated by equation (2) or (3) takes a 
higher value as centroid Gn of the candidate region approaches centroid O 
of the image area of the image information. Alternatively, calculation may 
be employed in which evaluation value P n takes a smaller value as centroid 
Gn of the candidate region approaches centroid O of the image area of the 
image information. In this case, the candidate region located close to the 
centroid of the image area of the image information will be extracted as the 
specific region by setting the predetermined condition to a condition that 
evaluation value P n is below a certain value. Furthermore, another 
evaluation value may be employed as long as it is an evaluation value of a 
candidate region exhibiting monotone increase or monotone decrease in 
correlation to the relative position of a candidate region with respect to the 
image area of the image information. 

[0 102] Referring to Fig. 13D, it is assumed that the evaluation values 
of centroids Go - G4 of each candidate region has been calculated as Po = 50, 
Pi = 90, P 2 = 80, P 3 = 70, P 4 = 50, respectively, by equation (2). 
[0103] In the case where the predetermined condition is preset with 
evaluation value P n exceeding the threshold value of 60, the two rule mark 
regions 513 corresponding to evaluation values Po and P4 are not extracted 
as specific regions. The two text regions 511 and diagram region 512 
corresponding to evaluation values Pi, P2 and P3 are extracted as specific 
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regions. 

[0104] The predetermined condition may be a condition defined by the 
relative position of a candidate region with respect to the image area of 
image information and an attribute of the candidate region. For example, 
by multiplying evaluation value P n calculated from the relative position of a 
candidate region with respect to image information by a factor K 
corresponding to the attribute of the candidate region, i.e., P n " = Kx P n , a 
candidate region having an evaluation value P n ' exceeding a predetermined 
threshold value can be extracted as a specific region. Specifically, when 
the factor K for a text region, a diagram region, a picture region, a rule 
mark region, and a margin region is set as Ki =1.0, K2 = 0.8, K3 = 0.8, K4 = 
0.6, and Ks = 0, respectively, evaluation value P n ' is established as Po' = 50 
x K 4 = 30, Pi 1 = 90 x Ki = 90, P 2 ' = 80 x Ki = 80, Ps* = 70 x K 2 = 56, and P 4 ' = 
50 x K4 = 30, respectively. Therefore, under the predetermined condition 
of evaluation value P n ' calculated from the relative position of a candidate 
region with respect to image information exceeding the threshold value of 
50, for example, the two text regions 511 and diagram region 512 
corresponding to evaluation values Pi', P2', Pa' are extracted as the specific 
region. 

[0105] The predetermined condition may be a condition defined by the 
size of the candidate region. In this case, an evaluation value of a 
candidate region is calculated based on the size of the candidate region, and 
a specific region is extracted in accordance with the evaluation value. 
More specifically, an evaluation value of a candidate region exhibiting 
monotone increase or monotone decrease in correlation with the size of a 
candidate region is calculated, and a candidate region whose evaluation 
value satisfies the predetermined condition is extracted as the specific 
region. 

[0106] The predetermined condition may be a condition defined by the 
attribute and size of a candidate region, defined by the size of a candidate 
region and the relative position of a candidate region with respect to image 
area of image information, or defined by the attribute and size of a 
candidate region, and the relative position of a candidate region with 
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respect to an image area of image information. 

[0107] Figs. 15A-15D are diagrams to describe image processing carried 
out by image processing apparatus 100 according to a modification of the 
first embodiment. Fig. 15A represents a document 501. Document 501 
includes text information and diagram information. Document 501 is read 
in as image information through image input device 200 to be applied to 
image processing apparatus 100. Description is based on the case where 
document 501 is applied in an inclined state. 

[0108] Fig. 15B represents image information 502 previous to rotation. 
Image information 502 previous to rotation is input to image processing 
apparatus 100 as image information of the smallest rectangular area 
including test information and diagram information. A text region 511 
and a diagram region 512 are extracted as candidate regions from image 
information 502 previous to rotation, and inclination of the image 
information is detected. Then, centroid Gi (ai, bi) of text region 511, and 
centroid G2 (a2, b2) of diagram region 512 are obtained. Evaluation values 
Pi and P2 for respective centroids are calculated. In the present 
specification, it is assumed that evaluation value Pi exceeds a 
predetermined value, and evaluation value P2 does not exceed the 
predetermined value. Accordingly, text region 511 is extracted as a 
specific region whereas diagram region 512 is not extracted as a specific 
region. 

[0109] Fig. 15C represents rotated image information 503. When 
image information 502 previous to rotation is rotated by the angle of 
rotation corresponding to the detected inclination of Fig. 15B, detection is 
made whether text region 511 extracted as a specific region protrudes from 
the image area of image information 502 previous to rotation. In this case, 
text region 511 protrudes from the image area of image information 502 
previous to rotation. 

[01 10] Fig. 15D represents extracted image information 504. Based on 
the detection of protruding at Fig. 15D, the smallest rectangular area 
including text region 511 extracted as a specific region is defined as the 
extraction area. The extraction area is extracted as extracted image 
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information 504. 

[0 11 1] In the modification of the first embodiment, the relative position 
of a candidate region is obtained with the centroid of the image area of the 
image information as the reference point. The reference point is not 
limited to a centroid, and an arbitrary reference point can be defined as 
long as it is located in the image area of image information. 
[0112] In image processing apparatus 100 according to a modification 
of the first embodiment, a plurality of candidate regions that are candidates 
of a specific region are extracted from the input image information, and an 
appropriate candidate region is extracted as a specific region from the 
plurality of candidate regions based on the relative position of the 
candidate region with respect to the image area of the image information. 
As a result, the event of loosing information present at a specific relative 
position with respect to the image area of the image information from the 
image information subjected to rotational correction can be prevented. 

Second Embodiment 
[0113] In the previous first embodiment, extraction area determination 
unit 116 of image processing apparatus 100 determines an extraction area 
so as to include a specific region. In the second embodiment, an extraction 
area determination unit 116A of an image processing apparatus 100A 
determines an extraction area so that the relative position with respect to a 
specific region satisfies a predetermined condition. 

[0 1 14] The structure of an image processing apparatus of the second 
embodiment is similar to that of the first embodiment described with 
reference to Fig. 1. Therefore, description thereof will not be repeated. 
[0115] Fig. 16 is a functional block diagram of an image processing 
apparatus 100A Image processing apparatus 100A includes an image size 
reduction unit 111, an under color removal unit 112, a region separation 
unit 113, a specific region extraction unit 114, an image inclination 
detection unit 115, an extraction area determination unit 116A, an image 
rotation unit 117, and an image extracting unit 118. 
[0116] Image size reduction unit 111, under color removal unit 1 12, 
region separation unit 113, specific region extraction unit 114, image 
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inclination detection unit 115, image rotation unit 117 and image 
extracting unit 118 are similar to those of image processing apparatus 100 
of the first embodiment described with reference to Fig. 2. Therefore, 
description thereof is not repeated. 

[0117] An extraction area determination unit 116A assigns a score to a 
plurality of candidate areas having the same direction and size as the 
image area of image information previous to rotation, based on the relative 
position with respect to a rotated specific region in accordance with the 
inclination detected by image inclination detection unit 115 of a specific 
region extracted by specific region extraction unit 114. The candidate area 
exhibiting the best score is identified as the extraction area among the 
plurality of candidate areas assigned with scores. Determining an 
extraction area is equal to determining the position of an extraction area. 
[0118] Fig. 17 represents the relative position of a specific region with 
respect to a candidate area 505. When centroid O of the image area of the 
image information is taken as the origin, the coordinates of centroid G n of 
the specific region is represented as (a n , b n ). The length of the image area 
of the image information in the horizontal direction and vertical direction is 
represented as V and M h", respectively. The coordinates of centroid G of 
the candidate area to determine an extraction area is established as (x, y). 
An extraction area is determined by optimizing the position of centroid G of 
a candidate area so that the relative position of a candidate area with 
respect to a specific region satisfies a predetermined condition. 
[0 1 19] Image processing carried out by image processing apparatus 
100A of the second embodiment corresponds to a mo difi cation of the 
extraction area determination process executed at step S16 in the image 
processing procedure of image processing apparatus 100 of the first 
embodiment described with reference to Fig. 3. 

[0120] Fig. 18 is a flow chart of the extraction area determination 
processing carried out by image processing apparatus 100A of the second 
embodiment. Extraction area determination unit 116 of image processing 
apparatus 100A obtains the horizontal length w and vertical length h of the 
image area of the image information (step S71). The rotated coordinates 
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(a n . bn) of centroid G n of a specific region subjected to rotation are calculated 
(step S72). 

[0121] Evaluation value P n (x, y) with respect to each specific region for 
a candidate area is calculated (step S73). The evaluation value calculated 
at step S73 is an evaluation value for a candidate area. Then, an 
evaluation function P (x, y) that is the sum of evaluation values Pn (x, y) of 
each region calculated at step S73 is obtained (step S74). Specifically, an 
evaluation function P (x, y) for a candidate area represented by equation (4) 
is calculated at steps S73 and S74. 

p(x y) = y 100(w - 2 | x - a n l)(h = 2 j y - b n 1) _ (4) 

wh 

[0122] Then, determination is made whether optimization has ended 
for evaluation function P (x, y) (step S75). When determination is made 
that optimization has ended, control proceeds to step S77. When 
determination is made that optimization has not ended, the centroid (x, y) 
of the candidate area is altered (step S76), and control returns to step S73. 
In the present specification, optimization is to obtain the centroid (x, y) of a 
candidate area with the highest value of evaluation function P (x, y). 
[0 123] Evaluation function P (x, y) is not limited to that represented by 
equation (4). Any equation as long as the relative position of an image 
area with respect to a specific region is represented can be employed, such 
as equation (5) set forth below. 

P(x,y) = £(w-2|x-a n |)(h-2|y-b n |) —(5) 

n 

[0124] Finally, a candidate area with point (x, y) optimized at steps 
S73-S76 as the centroid is determined as the extraction area (step S77). 
[0125] In the extraction area determination process described with 
reference to Fig. 18, a candidate area whose relative position of an image 
area with respect to a specific region satisfies a predetermined condition is 
determined as the extraction area. Alternatively, a candidate area with 
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point (x, y) that optimizes evaluation function P' (x, y) determined by the 
relative position of the image area with respect to a specific region and the 
attribute of the specific region as the centroid can be determined as the , 
extraction area. Evaluation function P' (x, y) may employ, for example, 
equation (6). 

P»(x,y) = v- iOOKnCw^lx-an |)(h - 2 j y - b n |) _ ($) 
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where K n is a factor in accordance with the attribute of that specific region. 
For example, factor K n = 1.0 when the specific region is a text region. 
Similarly, the factors of K n = 0.8, K n = 0.8, K n = 0.6, and K n = 0 are 
established when the specific region is a diagram region, a picture region, a 
rule mark region, and a margin region, respectively. 
[0126] A candidate area with a point that optimizes the evaluation 
function defined by the relative position of an image area with respect to a 
specific region and the size of a specific region as the centroid may be 
determined as an extraction area. Alternatively, a candidate area with a 
point that optimizes the evaluation function defined by the relative position 
of an image area with respect to a specific region, the attribute of a specific 
region, and the size of a specific region as the centroid may be determined 
as the extraction area. 

[0127] Evaluation value P or P' calculated by equation (4), (5) or (6) 
attains the highest value when centroid G (x, y) of the extraction area is 
located at the best position with respect to the centroid G n (a n , bn) of each 
specific region. Conversely, an evaluation function having the smallest 
value when centroid G (x, y) of the extraction area is located at the best 
position with respect to centroid G n (a n , bn) of each specific region may be 
used. 

[0128] Figs. 19A-19D are first diagrams to describe image processing 
carried out by image processing apparatus 100A of the second embodiment. 
Fig. 19A represents a document 501. Document 501 includes text 
information at the upper left region. Document 501 is entered as image 
information through image input device 200 into image processing 



-27- 



apparatus 100A. Description is based on document 501 input in an 
inclined state. 

[0129] Fig. 19B represents image information 502 previous to rotation. 
Image information 502 previous to rotation, applied to image processing 
apparatus 100 A, includes text information. A specific region is extracted 
from image information 502 previous to rotation, and inclination of the 
image information is detected. In the present specification, the text region 
is extracted as the specific region in this specific region extraction process. 
[0130] Fig. 19C represents rotated image information 503. An 
evaluation value is assigned to a plurality of candidate areas having the 
same direction and size as image information 502 previous to rotation, in 
accordance with the relative position to a rotated specific region. A 
candidate area having the best evaluation value among the plurality of 
candidate area assigned with an evaluation value is determined as the 
extraction area. Here, a candidate area with a point (x, y) that optimizes 
evaluation function P (x, y) represented by equation (4) described with 
reference to Fig. 19 as the centroid is defined as the extraction area. In 
the case where there is one specific region, the candidate area whose 
centroid corresponds to the centroid of the specific region is defined as the 
extraction area. 

[0131] Fig. 19D represents extracted image information 504. The 
extraction area defined by Fig. 19C is extracted as extracted image 
information 504. 

[0132] Figs. 20A-20D are second diagrams to describe image 
information carried out by image processing apparatus 100A of the second 
embodiment. Fig. 20 A represents a document 501. Document 501 
includes rule mark information, text information and diagram information. 
Document 501 is applied as image information by image input device 200 to 
be provided to image processing apparatus 100A. Description is based on 
document 501 entered in an inclined state. 

[0133] Fig. 20B represents image information 502 previous to rotation. 
Image information 502 previous to rotation, applied to image processing 
apparatus 100 A, includes rule mark information, text information and 
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diagram information. A specific region is extracted from image 
information 502 previous to rotation, and inclination of image information 
is detected. In the present specific region extraction process, a rule mark 
region, text region, and diagram region are extracted as specific regions. 
[0134] Fig. 20C represents rotated image information 503. An 
evaluation value is assigned to a plurality of candidate areas having the 
same direction and size as image information 502 previous to rotation in 
accordance with the relative position with respect to the rotated specific 
region. A candidate area exhibiting the best evaluation value among the 
plurality of candidate areas assigned with an evaluation value is defined as 
the extraction area. Here, a candidate area with point Gb (xb, yB) 
optimizing evaluation function P' (x, y) represented by equation (6) 
described with reference to Fig. 19 as the centroid is defined as the 
extraction area. Specifically, evaluation value Pb (xb, yB) with respect to 
centroid Gb (xb, yB> of candidate area has a higher value than evaluation 
value Pa (xa, yA) for centroid Ga (xa, yA> of another candidate area. The 
another candidate area has the text region and diagram region partially 
protruding. In contrast, the text region and diagram region will not 
protrude in candidate area. However, a portion of the rule mark area 
protrudes. Therefore, candidate area is defined as the extraction area. 
Thus, loosing important information such as text information and diagram 
information can be prevented. 

[0135] Fig. 20D represents extracted image information 504. The 
extraction area defined at Fig. 20C is extracted as image information 504. 
[0 136] In image processing apparatus 100A of the second embodiment, 
an evaluation value (score) is assigned to a plurality of candidate areas 
having the same direction and size as the image area of image information 
previous to rotation, in accordance with the relative position to a specific 
region from the rotated image information. The candidate area exhibiting 
the best evaluation value (score) is extracted among the plurality of 
candidate areas assigned with an evaluation value (score). Therefore, 
loosing important information from image information subjected to rotation 
correction can be prevented. Also, the arrangement of information 
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included in the image information subjected to rotation correction can be 
optimized. As a result, the specific region is arranged at the center of the 
extraction area when there is only one specific region. In the case where 
there are a plurality of specific regions, an extraction area having the best 
relative position of a plurality of specific regions with respect to the 
extraction area is extracted. 

[0137] The second embodiment is based on the description of processing 
carried out by image processing apparatus 100 A. The present invention 
can be taken as an image processing method executing a process of the 
extraction area determination process of Fig. 18 carried out at step S16 in 
the process of Fig. 3, an image processing program causing a computer to 
execute the process of an extraction area determination process of Fig. 18 at 
step S16 of the process of Fig. 3, and a computer readable recording 
medium recorded with an image processing program. 
[0138] Although the present invention has been described and 
illustrated in detail, it is clearly understood that the same is by way of 
illustration and example only and is not to be taken by way of limitation, 
the spirit and scope of the present invention being limited only by the terms 
of the appended claims. 
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